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(54) SOUND GENERATOR AND VIDEO GAME MACHINE EMPLOYING IT 

(57) When any of push-button switches on a con- 
troller 40 is pressed in a sound input mode, a video FI < 
game machine body 10 generates and temporarily 
stores frequency data of a tone corresponding to the 
pressed switch. At this time, when a joystick 45 on the 
controller 40 is tilted to a predetermined direction, the 
video game machine body 10 changes the generated 
frequency data according to the amount of tilt of the joy- 
stick 45. It is therefore possible to input various sounds 
in tone using a limited number of switches. The fre- 
quency data stored in the video game machine body 10 
is read later to be converted into audio signals, and out- 
putted from a speaker incorporated in a CRT display 30. 
When a melody based on the inputted sound coincides 
with a melody set in advance, the video game machine 
body 10 makes various changes in the progress of the 
game. For example, a hero character is warped to a 
position that is different from the present position, or 
provided with various items. 
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Description 

TECHNICAL FIELD 

[0001] The present invention relates to sound gen- 
erating devices and video game devices using the same 
and, more specifically, to a sound generating device 
which plays music based on tone data inputted with a 
video game machine controller and a video game 
device usi ng music play based on inputs from a player in 
relation to the progress of a game. 

BACKGROUND ART 

[0002] As a conventional example of video games 
that generate sound (or music), a game software "Mario 
Paint" has been marketed by the Applicant. In "Mario 
Paint", a musical staff is displayed on a screen. Symbols 
for specifying notes, tone qualities, or the like are written 
in the musical staff by operating a controller, and 
thereby a sound to be generated is inputted. In another 
example of video games that generate sounds, when a 
switch for specifying an operation or motion such as 
missile firing, jump, and punch, is pressed, a sound 
effect corresponding to that operation or motion (missile 
firing sound, sound effects representing jump, punch, or 
the like) is generated based on a program. In still 
another example, BGM is generated in accordance with 
changes in game screen. Further, conventional exam- 
ples of electronic toys that deal with sound include an 
electronic musical instrument (keyboard instrument) 
with a keyboard having key switches corresponding to 
tones. 

[0003] As described above, sound generating 
devices lor use in the conventional video games (includ- 
ing video games for a game-dedicated machine and for 
a personal computer) and the like are required to dis- 
play a musical staff. This requirement makes the pro- 
gram complicated. Also, operation of inputting sounds 
or notes is not easy, and these devices are not the type 
generating the sound of the tone according to key input 
by a player. Further, the electronic instruments with a 
keyboard can generate only the sound that corresponds 
to the switch being pressed. Therefore, such instru- 
ments require key switches as many as the tones in a 
required range, and it is difficult to input sounds with a 
small number of switches. For complicated sound varia- 
tion, these electric instruments become complicated in 
construction and thus expensive. Furthermore, in the 
conventional video games with a sound generating 
function, sound or music generated through the opera- 
tion by the player cannot change or have an effect on 
the progress of the game. 

[0004] Therefore, an object of the present invention 
is to provide a sound generating device enabling gener- 
ation of sounds of tones or music that cannot be 
expressed with a limited small number of switches. 
[0005] Further, another object of the present inven- 



tion is to provide a sound generating device enabling 
generation of sounds of a complicated scale or music 
with a simple construction. 

[0006] Still another object of the present invention is 
5 to provide a video game device enabling a player to 
input sounds and play music at will with a game 
machine controller having a small number of switches, 
and to use the music in relation to the progress of a 
game. 

10 

DISCLOSURE OF THE INVENTION 

[0007] Further, it is possible to realize a video game 
device enabling a player to input sounds and play music 
is at will with a game machine controller and also to relate 
the sounds or music to the progress of the game. That 
is, it is possible not only to generate a sound by pressing 
a button but also to finely adjust a tone through the 
operation of a joystick, thereby allowing generation of 
20 various sounds or music at will. 

[0008] To achieve the above objects, the present 
invention has characteristics as described below. 
[0009] A first aspect of the present invention is 
directed to a sound generating device to which sounds 
25 of different tones are inputted and generating the input- 
ted sounds by specifying the tones with a controller hav- 
ing a plurality of push-button switches and an analog 
joystick capable of selecting among a plurality of posi- 
tions, comprising: 

30 

a push-button detection part detecting one of the 
plurality of push-button switches that is pressed; 
a tone selection part selecting a tone correspond- 
ing to the push-button detected by the push-button 
35 detection part; 

a tilt amount detection part detecting an amount of 
tilt of the analog joystick, 

a frequency generation part generating a frequency 
corresponding to the tone selected by the tone 
40 selection part with or without change, based on the 
amount of tilt detected by the tilt amount detection 
part and the push-button switch detected by the 
push-button detection part; and 
an audio signal generation part generating a signal 
45 of a sound of the tone corresponding to the fre- 
quency generated by the frequency generation 
part. 



[0010] As described above, in accordance with the 
so first aspect, the audio signal having the frequency corre- 
sponding to the pressed push-button is generated with 
or without change. Therefore, it is possible to generate 
sounds (or music) of different tones using a limited 
number of push-button switches. 
55 [0011] According to a second aspect, in the first 
aspect, 

when the tilt amount detection part does not detect 
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the amount of tilt of the analog joystick, the fre- 
quency generation part generates the frequency 
corresponding to the tone selected by the tone 
selection part without change, and 
when the tilt amount detection part detects the 
amount of tilt of the analog joystick, the frequency 
generation part generates the frequency corre- 
sponding to the tone selected by the tone selection 
part with change according to the detected amount 
of tilt. 

[0012] As described above, in accordance with the 
second aspect, the frequency of the audio signal corre- 
sponding to the pressed push-button is changed 
according to the amount of tilt of the analog joystick 
Therefore, adjusting the amount of change is easy. 
[0013] According to a third aspect, in the first 
aspect, 

the frequency generation part comprises: 

a frequency data generation part generating 
frequency data corresponding to the push-but- 
ton switch of the tone selected by the tone 
selection part; 

a frequency data storage part temporarily stor- 
ing a plurality of frequency data; and 
a read/write part reading the frequency data 
stored in the frequency data storage part or 
writing the frequency data generated by the fre- 
quency data generation part in the frequency 
data storage part, 

when the tilt amount detection part does not detect 
the amount of tilt of the analog joystick, the 
read/write part writes in the frequency data storage 
part a digital value equivalent to the frequency cor- 
responding to the tone selected by the tone selec- 
tion part, as the frequency data; and 
when the tilt amount detection part detects the 
amount of tilt of the analog joystick, the read/write 
part writes in the frequency data storage part a dig- 
ital value equivalent to a frequency obtained by 
changing the frequency corresponding to the tone 
selected by the tone selection part according to the 
detected amount of tilt as the frequency data. 

[0014] As descrfoed above, in accordance with the 
third aspect, the frequency data corresponding to the 
pressed push-button switch with or without change is 
temporarily stored in the frequency data storage part, 
and later read out for use. Therefore, it is not required to 
operate an operation part in real time according to 
music play, thereby allowing easy operation to specify 
tones. 

[0015] According to a fourth aspect in the first 
aspect, 



the frequency generation part 

raises the frequency of the tone within a prede- 
termined tone range as the analog joystick is 
5 tilted to one direction; and 

lowers the frequency of the tone within a prede- 
termined tone range as the analog joystick is 
tilted to another direction. 

10 [0016] As described above, in accordance with the 
fourth aspect, the frequency of the tone is raised or low- 
ered according to the tilting direction of the analog joy- 
stick. This enables the operator to intuitively relate the 
changing directions of the analog joystick and the fre- 
15 quency of the tone to each other and therefore to easily 
perform operation for changing the frequency. 
[001 7] According to a fifth aspect, in the first aspect, 

the sound generating device further comprises a 
20 vibrato part for changing a depth value of vibrato 
according to the amount of tilt detected by the tilt 
amount detection part, 

the frequency generation part generates a fre- 
quency corresponding to the tone selected by the 
25 tone selection part with vibrato added thereto 
based on the depth value from the vibrato part. 

[0018] As described above, in accordance with the 
fifth aspect, the depth value of vibrato added to the 
30 sound of the selected tone is changed according to the 
amount of tilt of the analog joystick. Therefore, it is pos- 
sible to realize quite amusing sound effects. 
[0019] A sixth aspect is directed to a sound gener- 
ating device to which sounds of different tones are 
35 inputted and generating music based on the inputted 
sounds by specifying the tones with a controller having 
a plurality of push-button switches and an analog joy- 
stick capable of selecting among a plurality of positions, 
comprising: 

40 

a push-button detection part detecting one of the 
plurality of push-button switches that is pressed; 
a tone selection part selecting a tone correspond- 
ing to the push-button detected by the push-button 
45 detection part; 

a tilt amount detection part detecting an amount of 
tilt of the analog joystick; 

a frequency data generation part generating fre- 
quency data corresponding to the tone selected by 
so the tone selection part with or without change, 
based on the amount of tilt detected by the tilt 
amount detection part and the pressed push-button 
switch detected by the push-button detection part; 
a frequency data storage part temporarily storing a 
55 plurality of frequency data; 

a write part periodically and sequentially writing the 
frequency data generated by the frequency data 
generation part in the frequency data storage part; 
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a read part for sequentially reading the frequency 
data stored in the frequency data storage part; and 
an audio signal generation part generating an audio 
signal having a frequency corresponding to the fre- 
quency data read by the read part. 5 

[0020] As described above, in accordance with the 
sixth aspect, the audio signal having the frequency cor- 
responding to the pressed push-button is generated 
with or without change. It is therefore possible to gener- w 
ate sounds of various, tones (or music) using a limited 
number of push-button switches. Further, the frequency 
of the audio signal corresponding to the pressed push- 
button is changed according to the amount of tilt of the 
analog joystick Therefore, the amount of change is eas- is 
ily adjusted. Still further, the frequency data correspond- 
ing to the pressed push-button switch with or without 
change is temporarily stored in the frequency data stor- 
age part, and later read out for use. Therefore, real time 
operation of the operation part according to music play 20 
is not required, allowing easy operation to specify tones 
even if the user is not accustomed to the operation of 
the operation part. 

[0021] According to a seventh aspect, in the sixth 
aspect, 2S 



a frequency data generation part generating fre- 
quency data corresponding to the tone selected by 
the tone selection part with or without change, 
based on the amount of tilt detected by the tilt 
amount detection part and the push-button switch 
detected by the push-button detection part; 
a frequency data storage part temporarily storing a 
plurality of frequency data; 
a write part periodically and sequentially writing the 
frequency data generated by the frequency data 
generation part in the frequency data storage part; 
a read part sequentially reading the frequency data 
stored in the frequency data storage part; 
an audio signal generation part generating an audio 
signal having a frequency corresponding to the fre- 
quency data read by the read part; and 
a display image changing part changing at least 
one of the image data for the player-object gener- 
ated by the player-object image data generation 
part and the image data for the non-player-object 
generated by the non-player-object image data 
generation part based on the audio signal gener- 
ated by the audio signal generation part to change 
at least one of display states of the player-object 
and the non -player-object. 



the read part repeatedly reads the frequency data 
of a predetermined time period stored in the fre- 
quency data storage part to generate music com- 
posed by a player as BGM. 

[0022] As described above, in accordance with the 
seventh aspect, the data of the inputted tones can be 
used as BGM. 

[0023] An eighth aspect is directed to a video game 
device displaying an image on a display device and pro- 
ducing sound from a speaker by executing a game pro- 
gram, comprising: 

an operation part having a plurality of push-button 
switches for instructing motion of a player-object on 
a screen of the display device, and an analog joy- 
stick capable of selecting among a plurality of posi- 
tions and for instructing a moving direction of the 
player-object through operation; 
a player-object image data generation part generat- 
ing data for displaying an image of the player- 
object; 

a non-player-object image data generation part 
generating data for display an image of an object 
except the player-object; 

a push-button detection part detecting one of the 
plurality of push-button switches that is pressed; 
a tone selection part selecting a tone correspond- 
ing to the push-button detected by the push-button 
detection part; 

a tilt amount detection part detecting an amount of 
tilt of the analog joystick; 



[0024] As described above, in accordance with the 
eighth aspect, the data of the inputted sound can be 
used in relation to the progress of the game, thereby 
30 achieving an unprecedented amusing video game. 
[0025] According to a ninth aspect, in the eighth 
aspect, 



35 



40 



45 



the display image changing part changes the dis- 
play state of the non-player-object. 

[0026] According to a tenth aspect, in the ninth 
aspect, 

the display image changing means changes the 
display state of the non-player-object by moving the 
player-object to a scene which differs from a 
present scene to change a background screen of 
the player-object. 



[0027] As described above, in accordance with the 
tenth aspect, the display state of the non-player-object 
can be changed by warping the player-object to another 
position, for example. 
so [0028] According to an eleventh aspect, in the 
eighth aspect. 



55 



the display image changing part changes the dis- 
play state of the player-object. 

[0029] As described above, in accordance with the 
eleventh aspect, it is possible to change the display 
state of the player-object so that, for example, a hero 



>OC\D: <EP 1041536A1_I_> 



4 



7 



EP 1 041 536 A1 



8 



character can obtain various items (weapon, key, life, 
and the like). 

[0030] According to a twelfth aspect, in the eighth 
aspect, 

5 

the video game device further comprises a prede- 
termined melody determination part determining 
whether a melody based on the frequency data 
sequentially read from the read part is a predeter- 
mined melody, and 10 
the display image changing part changes at least 
one of the display states of the player-object and 
the non-player -object in response to determination 
by the predetermined melody determination part 
that the melody is the predetermined melody. is 

[0031] As descrfoed above, in accordance with the 
twelfth aspect, at least one of the display states of the 
player-object and the non-player-object is changed only 
when the melody based on the inputted sounds is a pre- so 
determined melody It is thus possible to include a mel- 
ody as an important factor for the progress of the game. 
[0032] According to a thirteenth aspect, in the 
twelfth aspect, 

25 

the predetermined melody determination part tem- 
porarily stores melody data inputted through opera- 
tion of the operation part; when new melody data is 
inputted through an operation of the operation part 
a predetermined time behind, compares the new 30 
melody data with the melody data previously input- 
ted; and when both data has a predetermined rela- 
tion, determines that the melody based on the 
frequency data sequentially read by the read part is 
the predetermined melody. 35 

[0033] As descrbed above, in accordance with the 
thirteenth aspect, the melody data inputted through the 
operation of the operation part is temporarily stored, 
and later read out for use. Therefore, real time operation 40 
of the operation part according to music play is not 
required, allowing easy operation to specify tones even 
if the user is not accustomed to the operation of the 
operation part. 

[0034] A fourteenth aspect is directed to a video 45 
game device displaying an image on a display device 
and producing sound from a speaker by executing a 
game program, comprising: 

an operation part operated by a player and having a so 
plurality of push-button switches for instructing 
motion of a player-object on a screen of the display 
device; 

a player-object image data generation part generat- 
ing data for displaying an image of the player- 55 
object; 

a non-player-object image data generation part 
generating data for displaying an image of an object 



except the player-object; 

a push-button detection part detecting one of the 
plurality of push-button switches that is pressed; 
a tone selection part selecting a tone correspond- 
ing to the push-button detected by the push-button 
detection part; 

a frequency data generation part generating fre- 
quency data corresponding to the tone selected by 
the tone selection part; 

a frequency data storage part temporarily storing a 

plurality of frequency data; 

a write part for periodically and sequentially writing 

the frequency data generated by the frequency 

data generation part in the frequency data storage 

part; 

a read part for sequentially read the frequency data 
stored in the frequency data storage part; 
an audio signal generation part generating an audio 
signal having a frequency corresponding to the fre- 
quency data read by the read part; and 
a display image changing part, based on the audio 
signal generated by the audio signal generation 
part, changing at least one of the display states of 
the player-object and the non-player-object by 
changing at least one of the image data for the 
player-object generated by the player-object image 
data generation part and the image data for the 
non-player-object generated by the non-player- 
object image data generation part 

[0035] As described above, in accordance with the 
fourteenth aspect, the data of the inputted sound can be 
used in relation to the progress of the game, allowing an 
unprecedented amusing video game. 
[0036] According to a fifteenth aspect, in the four- 
teenth aspect, 

the display image changing means changes the 
display state of the non-player-object by moving the 
player-object to a scene which differs from a 
present scene to change a background screen of 
the player-object. 

[0037] As described above, in accordance with the 
fifteenth aspect, the display state of the non-player- 
object can be changed by warping the player-object to 
another position, for example. 

[0038] A sixteenth aspect is directed to a recording 
medium in which a video game program to be executed 
by an information processing device for displaying an 
image for a game on a display device and producing 
sound for the game from a speaker is stored, 

the information processing device comprising an 
operation part operated by a player and having a 
plurality of push-button switches for instructing 
motion of a player-object on a screen of the display 
device, 
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the video game program being or realizing an oper- 
ational environment on the information processing 
device, the program comprising the steps of: 
generating data for displaying an image of the 
player-object in response to an operation of the s 
operation part; 

generating data for displaying an image of an object 
except the player-object (non-player-object) in 
response to an operation of the operation part- 
detecting one of the plurality of push-button io 
switches that is pressed and selecting a tone corre- 
sponding to the pressed push-button; 
generating frequency data corresponding to the 
selected tone; 

generating an audio signal having a frequency cor- rs 
responding to the frequency data; and 
changing at least one of display states of the player- 
objects and the non-player-object by changing at 
least one of the image data for the player-object 
and the image data for the non-player-object so 

[0039] As described above, in accoidance with the 
sixteenth aspect, the game program which uses the 
data of the inputted sound in relation to the progress of 
the game can be provided. 

25 

BRIEF DESCRIPTION OF THE DRAWINGS 



[0040] 



30 



FIG. 1 is a block diagram showing a functional con- 
figuration of a video game system provided with a 
sound generating device according to one embodi- 
ment of the present invention. 
FIG. 2 is an external view more specifically illustrat- 35 
mg the configuration of the video game system pro- 
vided with the sound generating device according 
to the embodiment of the present invention. 
FIG. 3 is a block diagram showing an electrical con- 
figuration of the video game system shown in FIG. to 

FIG. 4 is a block diagram showing a controller 40 
and a RAM cartridge 50 shown in FIG. 2 in detail 
FIG. 5 rs a memory map illustrating memory space 
of external ROM 21 shown in FIG. 3. « 
FIG. 6 is a memory map showing part (image dis- 
play data area 24) of the memory space of the 
external ROM 21 in detail. 

of RAWMs mem ° ry map i,lustratin 9 memory space 

FIG. 8 is a flow chart of a main routine showing a " 
general operation of a game machine body 10 
shown in FIG. 2. 

FIG. 9 is a subroutine flow chart showing a detailed 
operation of player-object processing (step S3) ss 
shown in FIG. 8. 

FIG. 10 is a subroutine flow chart showing a 
detailed operation of background object processing 



(step S4) shown in FIG. 8. 
FIG. 1 1 is a subroutine flow chart showing part of 
detailed operation of sound processing (step 5) 
shown in FIG. 8. ; 
FIG. 12 is a subroutine flow chart showing the 
remaining part of the detailed operation of the 
sound processing (step 5) shown in FIG. 8. 
FIG. 13 is a diagram illustrating the whole three- 
dimensional space in one stage or field. 
FIG. 14 is a diagram exemplarily illustrating a dis- 
play of a melody selection screen. 
FIG. 15 is a diagram exemplarily illustrating a dis- 
play of a sound input screen. 
FIG. 16 is a diagram exemplarily illustrating a dis- 
play of an auto play screen. 
FIG. 17 is a diagram exemplarily illustrating a dis- 
play of a musical staff and notes in the sound input 
screen. H 

FIG. 18 is a diagram illustrating how the notes on 
the musical staff shown in FIG. 17 change accord- 
ing to key input operation. 
FIG 19 is a subroutine flow chart showing a 
detailed operation of auto play processing (step 
S530) shown in FIG. 12. 

FIG. 20 is a subroutine flow chart showing a 
detailed operation of free play processing (step 
S550) shown in FIG. 12. 

FIG 21 is a subroutine flow chart showing a 
detailed operation of game play processing (step 
S580) shown in FIG. 12. 

FIG. 22 is a diagram exemplarily showing a display 
of a notice board. 

FIG 23 is a subroutine flow chart showing a 
detailed operation of recording processing (step 
S585) shown in FIG. 21. 

FIG. 24 is a subroutine flow chart showing a 
detailed operation of drawing processing (step S7) 
shown in FIG. 8. 

FIG. 25 is a subroutine flow chart showing a 
detailed operation of audio processing (step S8) 
shown in FIG. 8. 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0041] FIG. 1 is a block diagram showing a func- 
tional configuration of a video game system provided 
with a sound generation device according to one 
embodiment of the present invention 
[0042] m FIG. 1 , the video game system according 
to the present embodiment has an unprecedented 
novel function of generating sounds in addition to a 
vrdeo game program executing function provided for 
conventional general video game systems. That is the 
video game system of the present embodiment speci- 
fies tones with the use of a game machine controller 
(operation part) having a plurality of push-button 
switches and an analog joystick capable of selecting 
among a plurality of positions (hereinafter abbreviated 
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as "joystick"), thereby inputting sound data of different 
tones and generating sounds (or music) based on the 
inputted sound data. 

[0043] in the video game system ol the present 
embodiment, a video game machine body, which per- 
forms various information processing, includes at least 
a push-button detection part, a tilt amount detection 
part, a frequency generation part, and an audio signal 
generation part. 

[0044] The push-button switches provided on the 
operation part of the game machine controller include, 
for example, switches for tone selection (switches for 
generating sounds "re", "fa", "la", "ti", and "re" that is an 
octave higher than the former), and auxiliary switches (a 
switch for raising the tone selected by the tone selection 
switch by a semitone, a volume switch for turn up the 
volume, a switch for canceling a sound input mode to 
return to a game mode, for example). The joystick 
includes X-axis and Y-axis photointerrupters to resolve 
the amount of tilt of a lever in X-axis and Y-axis direc- 
tions and generate pulses in proportion to the amount of 
tilt. By supplying pulse signals generated by these pho- 
tointerrupters to counters to count these signals, the 
counters generate count values in proportion to the 
amount of tilt of the joystick. 

[0045] The push-button detection part detects one 
switch that is pressed from among the plurality of push- 
button switches. The tone selection part selects a tone 
corresponding to the push-button detected by the push- 
button detection part. The tilt amount detection part 
detects the amount of tilt of the joystick. 
[0046] More specifically, the tilt amount detection 
part detects a tilt angle of the joystick from a neutral 
position toward a first direction on a scale of 64, for 
example. When determining based on the amount of tilt 
detected by the tilt amount detection part that the joy- 
stick is located at a neutral position (home position), the 
frequency generation part generates a frequency corre- 
sponding to the tone selected by the tone selection part 
without any change. On the other hand, when determin- 
ing based on the amount of tilt detected by the tilt 
amount detection part that the joystick is located at a 
position exclusive of the neutral position, the frequency 
generation part generates a frequency corresponding to 
the tone selected by the tone selection part with 
changes according to the amount of tilt of the analog 
joystick. The audio signal generation part generates a 
signal of the sound of the tone corresponding to the fre- 
quency generated by the frequency generation part. 
The signal outputted from the audio signal generation 
part is supplied to a sound producer such as a speaker, 
which produces the inputted sound. 
[0047] It is thus possible to input sounds or tones 
with easy operation by using a game machine control- 
ler. 

[0048] The video game machine body is provided 
with a vibrato part for generating a variable vibrato 
sound with easy operation, as required. This vibrato 



part changes a depth value of vibrato according to the 
amount of tilt detected by the tilt amount detection part. 
That is, when the joystick is tilted to a second direction 
which is different from the above first direction (for 

5 example, if the first direction for changing frequency is 
up/down, the second direction for detecting vibrato is 
selected to right/left), the vibrato part changes the depth 
value of vibrato according to the amount of tilt to the 
second direction. In this case, when determining based 

10 on the amount of tilt detected by the tilt amount detec- 
tion part that the joystick is located at the neutral posi- 
tion, the frequency generation part generates a 
frequency corresponding to the tone selected by the 
tone selection part without vibrato. On the other hand, 

is when determining based on the amount of tilt detected 
by the tilt amount detection part that the joystick is 
located at a position exclusive of the neutral position, 
the frequency generation part generates a frequency 
corresponding to the tone selected by the tone selection 

20 part with variation added thereto according to the depth 
value of vibrato (frequency of the sound with vibrato). 
[0049] Furthermore, when the joystick is tilted to a 
certain direction, the amounts of tilt to the first direction 
(up/down) for specifying the frequency and to the sec- 

25 ond direction (right/left) for specifying the depth value of 
vibrato may be resolved and detected, and the amount 
of change in frequency and the depth value of vibrato 
may be simultaneously specified. Furthermore, an 
attenuation part and/or a volume part may be provided 

30 to enhance vibrato effects. The attenuation part is used 
for gradually turning down the volume at predetermined 
time intervals to smoothly attenuate the volume to 0 
when the push-button switch is pressed. The volume 
part is used for adjusting the volume. 

35 [0050] The frequency generation part is con- 
structed of, for example, a frequency data generation 
part, a frequency data storage part, and a write/read 
part. The frequency data generation part generates fre- 
quency data corresponding to the push-button switch of 

40 the tone selected by the tone selection part. The fre- 
quency data storage part temporarily stores the fre- 
quency data corresponding to the inputted sound or 
tone. The write/read part writes the frequency data gen- 
erated by the frequency data generation part in the fre- 

45 quency data storage part or reads the frequency data 
stored in the frequency data storage part. Further, when 
the tilt amount detection part does not detect the 
amount of tilt of the joystick, the write/read part writes a 
digital value equivalent to the frequency corresponding 

so to the tone selected by the tone selection part in the fre- 
quency data storage part as the frequency data. Then 
the tilt amount detection part detects the amount of tilt, 
the write/read part takes the frequency corresponding 
to the tone selected by the tone selection part as the ref- 

55 erence frequency, changes the reference frequency 
according to the amount of tilt, and writes the changed 
reference frequency in the frequency data storage part 
as the frequency data. The frequency data read by the 
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write/read part from the frequency data storage part is 
converted by the audio signal generation part into an 
audio signal having a frequency corresponding to the 
frequency data. " 

[0051] The video game machine body is further 
provided with, as required, a player-object image data 
generation part, a non-player-object image data gener- 
ation part, and a display image changing part The 
player-object image data generation part generates 
data for displaying an image of a player-object (for 
example, a hero character) to be operated by the player 
The non-player-object image data generation part gen- 
erates data for displaying an image of a non-player- 
object (for example, a background screen, still object 
and enemy object) that cannot be operated by the 
player. The display .mage changing part changes at 
least one of a display state of the player-object gener- 
ated by the player-object .mage data generation part 
and a display stale of the non-piayer-object generated 
by the non-player-object •mage data generation part 
according to the music generated by the audio signal 
generation part. 

10052] Possible specific examples of changing the 
display state include changing the display state of the 
non-player-object, changing the display state of the 
player-object, and combinations of both. For changing 
the display state of the non -player-object, various meth- 
ods can be thought. In one method, the background 
screen where the player-object is present is changed so 
as to proceed (or to be warped) to another scene or 
stage that differs from the preceding one. In one method 
for changing the display state of the player-object, when 
the player-object obtains an item such as a weapon 
plate armor, helmet, part of the player-object image is 
changed so that the player-object wears the obtained 
item. 

[0053] It is therefore possible to change at least one 
o the display states of the player-object and the non- 
player-object according to the music inputted by the 
player , thereby allowing more fun in game. 
[0054] Described next is a more specific embodi- 
ment of a video game system provided with the sound 
generating device according to the preferred embodi- 
ment of the present invention. Note that, although 
descnbed in the following embodiment is a case where 
the sound generating device of the present embodiment 
is applied to the video game system, the sound gener- 
ating device of the present embodiment can be applied 
to other information processing devices such as per- 
sonal computers and electronic musical instruments 
Furthermore, although the controller is a video game 
machine controller in the case described below the 
controller may take any structure as long as it has a plu- 
devfce° f SWitCheS ^ an ^'OS-type operation input 

[0055] FIG. 2 is an external view showing a more 
specific configuration of the video game system pro- 
vided with the sound generating device according to the 



embodiment of the present invention. In FIG 2 the 
video game system of the present invention is 'con- 
structed to include a video game machine body 10 a 
ROM cartridge 20, which is an example of external stor- 
5 age means, a CRT display 30. which is an example of a 
deplay device connected to the video game machine 
body 10, and a controller 40, which is an example of an 
operation part (or operation input part). A RAM car- 
tridge 50 (or a vibration cartridge 50A) is removably 
io attached to the controller 40. as required. 

[0056] The controller 40 is structured as such that a 
housing 41 having a shape that can be grasped by 
either one or both hands is provided with a plurality of 
switches or buttons. Specifically, the lower portions on 
is the left, center, and right of the housing 41 of the con- 
troller 40 are provided with handles 41 1_ 41 C, and 41 R 
respectively, and the upper surface thereof is'an opera- 
tional area. The operational area is provided at lower 
center with an analog joystick 45 capable of inputting 
so directions in an analog manner (hereinafter abbreviated 
as "joystick"). The operational area is further provided 
wrth a cross-shaped digital direction switch (hereinafter 
referred to as "cross switch") at left, and a plurality of 
button switches 47A to 47C at right. The joystick 45 is 
25 used for instructing or inputting a moving direction 
and/or a moving speed (or amount of movement) of the 
player-object according to the amount of tilt and direc- 
tion of the stick. Further, for sound input or music play 
through sound input, the joystick 45 is used in order to 
30 variously change the frequency of the generated sound 
by instructing the amount of change in frequency for 
changing the frequency of the inputted tone, or by spec- 
rfying a depth value indicating the depth of the sound 
when the sound is vibrated. The cross switch 46 is used 
35 in stead of or together with the joystick 45. for digitally 
instructing the moving direction of the player-object 
[0057] The plurality of button switches 47 includes 
switches 47A and 47B for instructing the motion of the 
player-object in a normal game mode, the switches 47C 
40 for use in switching the viewpoints of the image from a 
camera and other purposes, a motion switch 47L pro- 
vided on the upper-left side portion of the housing 41 a 
motion switch 47R provided on the upper-right side por- 
tion of the housing 41 . and a switch 472 provided on the 
45 backside of the handle 41 C. The switches 47C are 
formed of four button switches 47Cu, 47Cd, 47CI, and 
47Cr arranged in a cross. The switches 47C are used 
not only for switching the camera viewpoint, but also for 
controlling a moving speed and the like (for example 
so acceleration and deceleration) in a shooting game or an 
action game. 

[0058] Furthermore, in order to input an arbitrary 
sound or tone or to play music through sound input by 
using the video game machine controller 40. the switch 
55 47A is used as a button for selecting a tone (for exam- 
ple, a button which generates the sound of "re") The 
switch 47B is used for returning from a music play mode 
to the normal play mode. The switch 47R is used for 
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raising the selected tone by a semitone. The switch 47Z 
is used for turning up the volume (by 1.4 times, for 
example). The switches 47C (including the switches 
47Cu, 47CI. 47Cr, and 47Cd) are, like the switch 47A, 
used as buttons for selecting tones. In the embodiment 5 
described below, the switches 47Cd, Cr, CI, and Cu are 
used as buttons for specifying sounds "fa", "la", "ti", and 
"re" ("re" that is one octave higher than "re" of the switch 
47A), respectively. 
* [0059] Note that the functions of these switches 10 
47A to 47Z can be arbitrarily defined by a game pro- 
gram. 

' [0060] FIG. 3 is a block diagram showing the elec- 
trical configuration of the video game system shown in 
FIG. 2. In FIG. 3, the video game machine body 10 15 
incorporates a central processing unit (hereinafter 
abbreviated as "CPU") 11 and a reality coprocessor 
(hereinafter abbreviated as "RCP") 12. The RCP 12 
includes a bus control circuit 121 for bus control, an 
image processing unit (reality signal processor; herein- so 
after abbreviated as "RSP") 122 for polygon coordinate 
transformation, shading processing, and the like, and an 
image processing unit (reality display processor; herein- 
after abbreviated as "RDP") 123 rasterizing polygon 
data onto the image to be displayed and converting the 25 
results into those in a data format (dot data) storable in 
frame memory. A cartridge connector 13 into which the 
ROM cartridge 20 is removably inserted, a disk drive 
connector 14 into which a disk drive 26 is removably 
inserted, and RAM 15 are connected to the RCP 12. 30 
Further, an audio signal generator circuit 16 for output- 
ting an audio signal processed by the CPU 1 1 and an 
image signal generator circuit 17 for outputting an 
image signal processed by the CPU 1 1 are connected 
to the RCP 1 2. Further, a controller control circuit 1 8 for 35 
serially transferring operation data of one or more con- 
trollers (four controllers 40A to 40D are exemplarily 
shown in FIG. 3) and/or the data in the extended RAM 
cartridge 50 is connected to the RCP 12. 
[0061] The bus control circuit 121 included in the 40 
RCP 1 2 converts a command provided from the CPU 1 1 
through a bus as a parallel signal into a serial signal, 
and supplies the serial signal to the controller control 
circuit 18. The bus control circuit 121 also converts a 
serial signal from the controller control circuit 1 8 into a 45 
parallel signal, and supplies the parallel signal through 
the bus to the CPU 1 1 . The data indicating the operating 
states read from the controllers 40A to 40D is proc- 
essed by the CPU 11 or temporarily stored in the RAM 
15. In other words, the RAM 15 includes a storage area so 
for temporarily storing the data to be processed by the 
CPU 11, and used for smoothly reading or writing the 
data through the bus control circuit 121. 
[0062] A connector 1 95 provided on the rear side of 
the video game machine body 10 is connected to an ss 
output part of the audio signal generator circuit 16. A 
connector 196 provided on the rear side of the video 
game machine body 10 is connected to an output part 



of the image signal generator circuit 1 7. A sound pro- 
ducer 32 such as a television speaker is removably con- 
nected to the connector 195. A display 31 such as a 
television and CRT is removably connected to the con- 
nector 196. 

[0063] Controller connectors (hereinafter abbrevi- 
ated as "connectors") 191 to 194 provided on the front 
side of the video game machine body 1 0 are connected 
to the controller control circuit 1 8. The controllers 40A to 
40D are removably connected to the connectors 191 to 
194 through connecting jacks. As such, the controllers 
40A to 40D are connected to the connectors 191 to 194 
and, as a result, electrically connected to the video 
game machine body 10, thereby enabling transmission 
and transfer of data between these controllers and the 
video game machine body 10. 

[0064] FIG. 4 is a block diagram showing a detailed 
structure of the controller 40 and the RAM cartridge 50. 
In FIG. 4, the housing of the controller 40 accommo- 
dates various circuits such as an operation signal 
processing circuit 44 for detecting the operating states 
of the joystick 45, the switches 46 and 47, and others 
and transferring the detection data to the controller con- 
trol circuit 18. The operation signal processing circuit 44 
includes a receiver circuit 441, a control circuit 442, a 
switch signal detector circuit 443. a counter circuit 444, 
a transmitter circuit 445, a joyport control circuit 446, a 
reset circuit 447, and a NOR gate 448. The receiver cir- 
cuit 441 converts a serial signal such as a control signal 
transmitted from the controller control circuit 18 and 
data to be written in the RAM cartridge 50 into a parallel 
signal, and supplies the parallel signal to the control cir- 
cuit 442. When the control signal sent from the control- 
ler control circuit 18 is for resetting the X-Y coordinates 
of the joystick 45, the control circuit 442 produces a 
reset signal and supplies it to the counter 444 through 
the NOR gate 448. Thus, the counter values in an X- 
axis counter 444X and a Y-axis counter 444Y both 
included in the counter 444 are reset (to 0 forcefully). 
[0065] The joystick 45 includes X-axis and Y-axis 
photointerrupters for resolving the tilting direction of the 
lever into the X-axis direction and the Y-axis direction 
and generating pulses in proportion to the amount of tilt 
in each axis direction. These X-axis and Y-axis photoin- 
terrupters supply pulse signals to the X-axis counter 
444X and the Y-axis counter 444Y, respectively. When 
the joystick 45 is tilted in the X-axis direction, the X-axis 
counter 444X counts the number of pulses generated 
according to the amount of tilt. When the joystick 45 is 
tilted in the Y-axis direction, the Y-axis counter 444Y 
counts the number of pulses generated according to the 
amount of tilt. Therefore, a composite vector of X-axis 
and Y-axis defined by the counter values of the X-axis 
and Y-axis counters 444X and 444Y determines the 
moving direction and coordinate position of the player- 
object (main character, cursor, or the like). Note that the 
X-axis and Y-axis counters 444X and 444Y can be also 
reset by a reset signal supplied from the rest signal gen- 
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erator circuit 447 when powered on or by a reset signal 
supplied from the switch signal detector circuit 443 
when the player presses predetermined two switches 
simultaneously. At this time, each counter value is 
cleared too. 

[0066] Responding to an output command of switch 
states supplied from the control circuit 442 at predeter- 
mined intervals (for example, a 1/60 second interval 
which is a frame cycle for televisions), the switch signal 
detector circuit 443 reads a signal which varies accord- 
ing to the press states of the cross switch 46 and the 
switches 47A to 47Z, and supplies the signal to the con- 
trol circuit 442. Responding to a signal from the control- 
ler control circuit 18 for instructing read of the operating 
state data, the control circuit 442 supplies to the trans- 
mitter circuit 445 the operating state data of the 
switches 47A to 47Z and the counter values of the X- 
ax.s and Y-axis counters 444X and 444Y in a predeter- 
mined data format. The transmitter circuit 445 converts 
the parallel signal from the control circuit 442 into a 
serial signal, and transfers the serial signal to a con- 
verter circuit 43 and further to the controller control cir- 
cuit 18 through a signal line 42. The port control circuit 
446 is connected to the control circuit 442 through an 
address bus and a data bus. When the RAM cartridge 
50 is connected to a port connector 449, the port control 
circuit 446 controls output/input (or transmission/receiv- 
ing) of data according to instructions from the CPU 1 1 
[0067] The ROM cartridge 20 is constructed as 
such that its housing accommodates a substrate with 
external ROM 21 contained thereon. The external ROM 
21 stores image data and program data for image 
processing for game and the like, as well as audio data 
such as music, sound effects, and messages as 
required. 

[0068] FIG. 5 is a memory map illustrating memory 
space in the external ROM 21. FIG. 6 is a memory map 
showing part (image data area 24) of the memory space 
of the external ROM 21 in detail. As shown in FIG 5 the 
external ROM 21 includes, as storage areas, a program 
area 22, a character code area 23, an image data area 
24. and sound memory area 25. The external ROM 21 
prev.ously stores various programs therein in a fixed 
manner. 

[0069] The program area 22 stores programs 
requ.red for performing image processing on game and 
others and for realizing functions shown in flow charts 
(FIGS. 8 to 12, FIGS. 19 to 21. and FIGS. 23 to 25 
which will be described later), game data according to 
the game contents, and others. 
[0070] Specifically, the program area 22 includes 
storage areas 22a to 22i each for fixedly storing an 
operating program for the CPU 11 in advance In the 
mam program area 22a. a program for a main routine 
such as game processing is stored. In the control pad 
data (operating state) determination program area 22b 
a program for processing data indicative of the operat- 
ing state of the controller 40 and the like is stored In the 



write program area 22c. a write program to be executed 
when the CPU 1 1 instructs the RCP 1 2 to write data into 
frame memory and a Z buffer is stored. For example in 
the write program area 22c. a program for writing color 
5 data into a frame memory area (storage area 152 
shown in FIG. 7) of the RAM 15 and a program for writ- 
ing depth data into the Z buffer area (storage area 153 
shown in FIG. 7) are stored. Such color data and depth 
data are stored as image data based on texture data of 
io a plurality of moving objects or background objects to be 
displayed on a single background screen. In the moving 
object control program area 22d. a control program for 
chang.ng the position of the moving object in three- 
dimensional space by the RCP 12 under instructions 
rs from the CPU 1 1 is stored. In the camera control pro- 
gram area 22e. a camera control program for controlling 
from which position and in which direction moving 
objects including the player-object and background 
objects should be photographed. In the player-object 
so program area 22f, a program (refer to FIG. 9) for control- 
ling display of the object operated by the player is 
stored. In the background program area 22g a back- 
ground generation program (refer to FIG. 10) for gener- 
ating a three-dimensional background screen (still 
25 screen, course screen, or the like) by the RCP 1 2 under 
instructions from the CPU 11 is stored. In the audio 
processing program area 22h. a program (refer to FIG 
25) for generating sound effects, music and audio mes- 
sages is stored. In the game-over processing program 
30 area 22., a program for performing processing at the 
time of game-over (for example, detecting the state of 
game-over, and storing backup data of the game states 
that have been present before game-over) is stored. 
[0071] The character code area 23 is an area in 
35 which a plurality of types of character codes are stored 
For example, dot data of the plurality of types of charac- 
ters corresponding to codes are stored therein The 
character code data stored in the character code area 
23 is used for displaying a description for the player dur- 
40 mg the progress of the game. For example, the charac- 
ter codes are used for displaying appropriate operation 
at appropriate timing through messages (or lines) in 
character, according to environments surrounding the 
player-object (such as the place, the type of the obsta- 
45 de. and the type of the enemy-object) and the situation 
the player-object is experiencing. 
[0072] The image data area 24 includes storage 
areas 24a and 24b as shown in FIG. 6. For each back- 
ground object and/or moving object, image data such as 
so plural polygon coordinate data and texture data is 
stored in the image data area 24. Also in the image data 
area 24. a display control program is stored for fixedly 
displaying each object at a predetermined position or for 
displaying each object as it moves. For example in the 
55 storage area 24a. a program for displaying the player- 
object is stored. Further, in the storage area 24b a 
background object program for displaying a plurality' of 
background (or still) objects 1 to n1 is stored 
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[0073] In the sound memory area 25, sound data is 
stored, such as audio messages appropriate to each 
scene, sound effects, and game music. 
[0074] As the external storage device that is con- 
nected to the video game machine body 10, various 
storage media may be used such as CD-ROM and a 
magnetic disk, instead of or in addition to the ROM car- 
tridge 20. For using those media, the disk drive (record- 
ing/reproducing device) 26 is provided for reading or 
writing as required various game data (including pro- 
gram data and data for image display) from or into an 
optical or magnetic disk-like storage medium such as 
CD-ROM and a magnetic disk. The disk drive 26 reads 
data from the magnetic or optical disk in which program 
data similar to that in the external ROM 21 is optically or 
magnetically stored. The disk drive 26 transfers the read 
data to the RAM 15. 

[0075] FIG. 7 is a memory map illustrating memory 
space in the RAM 15. By way of example, the RAM 15 
includes, as storage areas, a display list area 150, a 
program area 151, a frame memory (or image buffer 
memory) area 1 52 for temporarily storing image data for 
one frame, the Z buffer area 153 for storing depth data 
for each dot of the image data stored in the frame mem- 
ory area, an image data area 154, a sound memory 
area 155. a storage area 156 for storing data of the 
operating state of a control pad, a work (working) mem- 
ory area 157. an audio list area 158, and a register/flag 
area 159. 

[0076] Each of the storage areas 151 to 159 is 
memory space accessible by the CPU 1 1 through the 
bus control circuit 121 or directly accessible by the RCP 
12. Arbitrary capacity (or memory space) is allocated to 
these areas according to the game in use. Part of the 
entire game program data for all stages (or called 
scenes or fields) stored in the storage areas 22, 24 and 
25 of the external ROM 21 is transferred and temporar- 
ily stored in the program area 1 51 , the image data area 
154, and the sound memory area 155, respectively 
(such part is, for example, a game program required for 
a certain stage or field in action games or role playing 
games (a course, in race games)). As such, by storing 
part of various program data required for a certain 
scene in each of the storage areas 151 , 154, and 155, 
the efficiency in processing can be increased, com- 
pared with reading such data directly from the external 
ROM 21 every time required by the CPU 1 1 . As a result, 
the image processing speed in the CPU 11 can be 
increased. 

[0077] Specifically, the frame memory area 1 52 has 
a storage capacity equivalent to (the number of picture 
elements (pixels or dots) of the display 30) x (the 
number of bits of color data per picture element), in 
which the color data corresponding to each picture ele- 
ment of the display 30 is stored per dot. In the frame 
memory area 152, the color data of the subject viewed 
from a viewpoint is temporarily stored per dot in the 
image processing mode, based on three-dimensional 



coordinate data. The three-dimensional coordinate data 
is to display one or more still object or moving objects 
stored in the image data area 154 to be displayed on a 
single background screen as a collection of plural poiy- 

5 gons. Also in the frame memory area 1 52, the color data 
for displaying various objects stored in the image data 
area 154 is temporarily stored per dot in the display 
mode. The various objects include moving objects such 
as a player-object, friend-object, enemy-object, and 

io boss-object, and background (or still) objects. Note that 
the moving objects such as an enemy-object and boss- 
object and the background (or still) objects cannot be 
moved or changed through operation of the controller 
40 by the player, and therefore may be generically 

75 called "non-player-objects w . 

[0078] The Z buffer area 1 53 has a storage capacity 
equivalent to (the number of picture elements (pixels or 
dots) of the display 30) x (the number of bits of depth 
data per picture element), in which the depth corre- 

20 sponding to each picture element of the display 30 is 
stored per dot. In the Z buffer area 153, the depth data 
of the subject viewed from a viewpoint is temporarily 
stored per dot in the image processing mode, based on 
three-dimensional coordinate data. The three-dimen- 

25 sional coordinate data is to display one or more still 
object or moving objects stored in the image data area 
154 to be displayed on a single background screen as a 
collection of plural polygons. Also in the Z buffer area 
153, the depth data of the moving and/or still objects is 

30 temporarily stored per dot in the display mode. 

[0079] In the image data area 154, coordinate data 
of the plurality of collections of polygons and texture 
data are stored for each still and/or moving object for 
game display stored in the external ROM 21. Prior to 

35 image processing operation, data for at least one stage 
or field is transferred to the image data area 154 from 
the external ROM 21. 

[0080] To the sound memory area 155, part of 
audio data (data of lines, music, and sound effects) 

40 stored in the external ROM 21 is transferred. In the 
sound memory area 155, the data transferred from the 
external ROM 21 is temporarily stored as data of sound 
to be generated from the sound producing device 32. 
Also in the sound memory area 1 55, sound or tone data 

45 inputted by the player is stored. In the audio list area 
158, audio data for creating sound to be produced by 
the speaker is stored. 

[0081 ] In the control pad data (operating state data) 
storage area 156, operating state data indicative of the 
so operating state read from the controller 40 is temporarily 
stored. In the work memory area 157, data such as 
parameters is stored during program execution by the 
CPU 11. 

[0082] The register/flag area 1 59 includes a register 
55 area 159R having a plurality of registers and a flag area 
159F having a plurality of flags. The register area 159R 
includes, for example, a melody data register R1 for 
storing tone data of a melody, a sound number register 
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R2 for storing the order of sounds, an input tone register 
R3 for storing the tone data inputted by the player a 
sound check register R4 for storing tone-check results 
and a the-number-of-background-objects register R5 for 
storing the number of background objects. The flag area 
159F is an area in which flags indicative of the states 
during game progress are stored. For example, a sound 
check flag Fl and game-over flag F2 for identifying the 
presence or absence of detection of the conditions for 
game-over are stored in the flag areas 159F. 
[0083] FIG. 8 is a flow chart of a main routine show- 
ing the general operation of the game machine body 10 
shown in FIG. 2. The operation of the present embodi- 
ment is described next according to the main routine 
flow chart of FIG. 8 with reference to detailed (or sub- 
routine) flow charts of respective operation shown in 
FIGS. 9 to 12, FIGS. 1 9 to 21 , FIGS. 23 to 25. 
[0084] When powered on, the video game machine 
body 10 is set to a predetermined initial state for start- 
ing. In response, the CPU 11 transfers a start-up pro- 
gram of the game program stored in the program area of 
the external ROM 21 to the program area 151 of the 
RAM 15, initializes each parameter, and then executes 
the main routine flow chart shown in FIG. 8. 
[0085] The main routine processing shown in FIG 8 
is performed by the CPU 1 1 for each frame (a 1/60 sec- 
ond). That is, the CPU 11 performs operations from 
steps S1 to S1 1 and then repeatedly performs operation 
from steps S2 to S1 1 until one stage (field, or course) is 
cleared. However, steps S7 and 8 are directly per- 
formed by the RCP 12. Further, the CPU 11 performs 
game-over processing of step S12 when the game is 
over without a success in stage clearing. On the other 
hand, when the stage is successfully cleared, the CPU 
1 1 returns from step S12 to step S1 . 
[0086] Specifically, in step S1, initialization for game 
start (that is, game start processing) is performed At 
this time, when the game can start from any point in the 
plural stages or courses, for example, a stage or course 
selection screen is displayed. However. Stage 1 of the 
game is played immediately after startup, and therefore 
game start processing for that stage is performed That 
is, the register area 159R and the flag area 159F are 
cleared, and various data required for playing Stage 1 
(or. selected stage or course) of the game is read from 
the external ROM 21 and transferred to the storaqe 
areas 151 to 155 of the RAM 15. 
[0087] Next, in step S2. controller processing is per- 
formed. In this processing, any one of the controllers 
that is operated among the joystick 45. the cross switch 
46. and the switches 47A to 472 is detected. Further in 
th.s processing, detection data (controller data) of an 
operating state is read and written. 
[0088] Next, in step S3, processing for displaying 
the player-object is performed. This processing is basi- 
cally to change the direction and shape of the player- 
obiect based on the operating state of the joystick 45 
operated by the player and the presence or absence of 



attacks from an enemy, which will be described later 
with reference to FIG. 9. In this display control of the 
player-object, for example, the coordinate position and 
shape of the polygon data of the player-object after 
5 change is calculated. This calculation is based on the 
program transferred from the storage area 22f, the poly- 
gon data of the player-object transferred from the stor- 
age area 24a, and the operating state of the joystick 45 
for example. As a result, a plurality of polygons are 
w obtained to compose a plurality of triangles. The color 
data is written into each address in the storage area 154 
corresponding to each surface of these triangles as if a 
pattern or a piece of color specified by the texture data 
is pasted. 

is [0089] Next, in step S4. processing for displaying 
the background object is performed. In this processing 
the display position and shape of the background object 
f emulated based on the program partially transferred 
from the storage area 22g and the polygon data of the 
so background object transferred from the storage area 24 
which will be described later with reference to FIG 10 
[0090] Next, in step S5, sound processing is per- 
formed. This processing is to produce music being 
played by the player, and its detail is shown in FIGS 1 1 
» and 1 2, which will be described later. Auto play process- 
ing in FIG. 12 is shown in detail in FIG. 19. Free play 
processing in FIG. 12 is shown in detail in FIG 20 

FIG 0 ^ 9 Pr0CeSS ' n9 in F,G 12 is shown in a" 6 *" 1 <n 
30 [0091] Next, in step S6. camera processing is per- 
formed. In this camera processing, for example the 
coordinates of each object are calculated when each 
object is viewed at a specified angle so that the line of 
sight or the field of view through the finder of the camera 
35 has the specified angle. 

[0092] Next, in step S7. the RSP 122 performs 
drawing processing. That is, the RCP 12 transforms 
image data of the moving and still objects for display 
(coordinate transformation processing and frame mem- 
40 ory drawing processing), based on the texture data of 
enemies, the player, background objects (moving and 
still objects) stored in the image data area 154 of the 
RAM 15. Specifically, the color data is written into each 
address in the storage area 154 corresponding to each 
45 01 PC/Son triangles for each of the moving and still 
objects so that a color and the like specified by the tex- 
ture data determined for each object is pasted The 
drawing processing will be described in detail with refer- 
ence to FIG. 24. 

so [0093] Next, in step S8, the audio processing is per- 
formed based on the audio data such as messages 
music, and sound effects. The audio data processing 
will be described in detail with reference to FIG 25 
[0094] Next, in step S9. the RCP 12 reads the 
55 image data stored in the frame memory area 1 52 based 
on the results of the drawing processing in step S7 and 
thereby the player-object, moving object, still object 
and enemy object and the like are displayed on the dis- 
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play screen 31. 

[0095] Next, in step S10, the RCP 12 reads the 
audio data obtained through the audio processing of 
step S8, and thereby audio such as music, sound 
effects, or speech is outputted. 

[0096] Next, in step S1 1 . whether the stage or field 
is cleared or not is determined (clear detection). If not 
cleared, whether the game is over or not is determined 
in step S1 1. If not over, the procedure returns to step 

• S2, and repeats the operations in steps S2 thorough 
S1 1 until the conditions for game-over are detected. 
Then, when the game-over conditions are detected, 

" such as, when the number of times allowed for the 
player to fail the stage or field reaches a predetermined 
value and when a predetermined number of lives of the 
player-object are consumed, predetermined game-over 
processing (processing to select either of continuing the 
game or not, processing to select either of storing 
backup data or not, and the like) is performed in step 
S12. 

[0097] On the other hand, when the conditions for 
stage clearing (such as beating the boss) is detected in 
step S1 1 , predetermined clear processing is performed 
in step S12, and then the procedure returns to step S1. 
[0098] The operation of each subroutine is now 
described below in detail. 

[0099] First, with reference to FIG. 9, the process- 
ing of displaying the player-object (step S3 in FIG. 8) is 
described in detail. In step S301 . joystick data stored in 
the control pad data area 156 is read and corrected. For 
example, data as to the center portion of an operable 
range of the joystick 45 is deleted. That is, the joystick 
data is processed to become "0" when the stick is posi- 
tioned at its home position, that is, in the vicinity of the 
center (a 1 0-count radius, for example). With such oper- 
ation, the joystick data in the vicinity of the center can be 
correctly controlled to "0" even when the joystick 45 has 
a manufacturing error or when the player's fingers 
slightly tremble. Further, data within a predetermined 
range in the vicinity of the periphery of the operable 
range of the joystick 45 is also corrected. This correction 
is made in order not to output data of unnecessary part 
during game progress. Next, joystick data Xj and Yj for 
use during the game are obtained. In other words, the 
data calculated in step S301 is represented by the count 
values of the X-axis counter 444X and the Y-axis coun- 
ter 444Y, and therefore these count values are con- 
verted into values that can be easily processed in the 
game. Specifically, Xj becomes "0" when the stick is not 
tilted, "+64" when tilted to maximum in -X direction (left- 
ward), and "-64" when tilted to maximum in +X direction 
(rightward). Yj becomes "0* when the stick is not tilted, 
"-64" when tilted to maximum in +Y direction (forward), 
and "+64" when tilted to maximum in -Y direction (down- 
ward). According to such joystick data, the coordinate 
position for moving the player-object is obtained. 
[0100] Next, in step S302, in response to push-but- 
ton switch operation, the processing is performed for 



controlling motions of the player-object (processing for 
making a motion such as jumping, cutting an enemy 
with a sword, and launching a missile). 
[0101] Next, in step S303, based on the data as to 

5 the player-object obtained in steps S301 and S302, the 
player-object data to be displayed on a single screen is 
registered in the display list area 150. This registration 
processing is performed as pre-processing for drawing 
processing (will be described later with reference to 

io FIG. 24) when the player-object is displayed. 

[0102] Next, with reference to FIG. 10, display 
processing of the background object (processing in step 
S4 of FIG. 8) is described in detail. In step S401 , 1 is set 
in the number-of -background-object register R5. Next, 

is in step S402, the background objects specified by the 
number-of-background-object register R5 is registered 
in the display list. Next, in step S403, the number-of- 
background-object register R5 is incremented by 1. 
Next, in step S404, it is determined whether processing 

20 for displaying all background objects set by the program 
has ended or not (in other words, whether the value in 
the number-of-background-object register R5 coincides 
with the number of background objects to be displayed 
on a single screen or not). If not yet ended, the proce- 
ss dure returns to step S402, and repeats the processing 
in steps S402 through S404. If ended, the procedure 
returns to step S5 of the main routine in FIG. 8. 
[0103] Here, prior to detailed description of sound 
processing (step S5 of FIG. 8), the game assumed in 

30 the present embodiment is briefly described. In the 
game, the player-object moves to various stages and 
fields in three-dimensional space to clear an event or to 
clear each stage by beating an enemy. During the 
game, the player operates the controller to input sounds 

35 or tones, and achieves the goal determined by the pro- 
gram while playing music. Further, in the game, one or 
more melodies are displayed on a notice board or the 
like during game play. When the player operates the 
controller for playing one of the melodies, it is deter- 

40 mined that the melody is a predetermined one (that is, a 
factor of changing the object). Accordingly, the display 
state of at least one of the player-object and the non- 
player-object is changed. 

[0104] In one specific example of object change 
45 when a predetermined melody or music is played (when 
sound is inputted), the player-object is moved (or 
warped) to a place in specific three-dimensional space. 
In another example, the player-object is allowed to enter 
a specific area (room) (or the player-object is made to 
so unlock the door). In other words, as for the former exam- 
ple, the background surrounding the player-object is 
changed to the background of the destination. As for the 
latter, the background surrounding the player-object is 
changed to the scene in that specific room. As such, the 
55 display state of the non-player-object is changed. In still 
another example of object change when a predeter- 
mined melody or music is played (when sound is input- 
ted), the player-object is allowed to unlock a jewelry box. 
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In still another example, the player-object is provided 
with a special item such as a protector or weapon. In 
these cases, the display state of non-player-objects is 
changed so that the jewelry box is opened, and the dis- 
play state is changed so that the player-object wears the s 
protector or carries the weapon. 
[0105] FIG. 13 shows the whole three-dimensional 
space in a single stage or field. However, FIG. 13 repre- 
sents the virtual world as a bird's eye view, and what is 
actually displayed on the screen of the CRT 30 as a io 
game screen is only part of the vicinity of the player- 
object. In this state, the player-object is at a lower-right 
position (place). When the player operates the controller 
40 and plays a predetermined melody, the player-object 
can move (or warp) to any one of first to third places cor- is 
responding to that melody. At this time, the camera pho- 
tographs the player-object after move and the 
background or still images in the vicinity of the player- 
object. As a result, the player-object and the back- 
ground in the vicinity thereof are displayed on the so 
screen of the CRT 30. 

[0106] Next, with reference to the subroutine flow 
charts of FIGS. 1 1 and 12, sound processing to be exe- 
cuted in step S5 of FIG. 8 is described in detail. In step 
S501, it is determined whether the melody selection 25 
screen is displayed or not. This melody selection screen 
is exemplarily shown in FIG. 14. When the player oper- 
ates (or clicks an icon marked by an instrument) a spe- 
cific button switch (for example, start switch 47S) to 
select a melody play mode (for example, a mode of so 
playing the ocarina), the melody selection mode is dis- 
played as a window. At this time, a list 305 of currently 
available melodies is displayed on the window. Also an 
alternative 302 of a free play mode (playing other mel- 
ody not included in the melody list), an alternative 303 of ss 
closing the window and the like are displayed on the 
window. Preferably, a musical score (not necessarily a 
musical staff) 304 is displayed on part of the window, 
and symbols of the switches corresponding to sounds 
or notes are displayed. The number of melodies 40 
included in the melody list 301 may be increased 
according to the progress of the game or event partici- 
pation during the game. The player operates the con- 
troller 40 to move upward or downward a cursor 305 
displayed on left on the window, thereby selecting an 45 
arbitrary melody, and also selecting a play mode or win- 
dow closing mode. In response, the CPU 1 1 executes 
the program corresponding to the selection. 
[0107] As described above, if it is determined in 
step S501 that the melody selection screen is displayed so 
it is determined in step S502 whether the player has 
selected the first melody (for example, melody of wind) 
or not. If it is determined that the first melody has been 
selected, data of the first melody is read in step S502 
from its storage location of the external ROM 21 or the ss 
program area 1 51 of the RAM 1 5, and then written into 
the melody data register R1. Then, in step S504, the 
value stored in the sound number register R2 is set to 



"I". Next, in step S505, check mode processing starts. 
Specifically, processing for switching the screen from 
the melody selection screen to a check mode screen 
(refer to FIGS. 15 and 16) is performed. Then, the pro- 
cedure returns to step S6 of the main routine shown in 
FIG. 8. 

[01 08] On the other hand, if it is determined in step 
S502 that the player has not selected the first melody, it 
is determined in step S506 whether the second melody 
(for example, melody of fire) has been selected by the 
player or not. If it is determined that the second melody 
has been selected, data of the second melody is read in 
step S507 from its storage location of the external ROM 
21 or the program area 151 of the RAM 15, and written 
into the melody data register R1. The procedure then 
advances to step S504. 

[01 09] If the selected melody is not the first or sec- 
ond melody, it is determined the n-th (n is an integer not 
less than 3 and not more than n max , and n max is a max- 
imum number defined by the program) melody has been 
selected or not. If it is determined that the n-th melody 
has been selected, data of the n-th melody stored in the 
external ROM 21 or the program area 151 of the RAM 
15 is read and written into the melody data register R1. 
Therefore, if it is determined in step S508 that the n max - 
th melody has been selected by the player, the n ^ -th 



melody data is written in step S509 in the melod/data 
register R1. The procedure then advances to steo 
S504. M 

[01 1 0] On the other hand, if it is determined in step 
S508 that any of the first to n^-th melody has not been 
selected, it is determined in step S510 whether the free 
play mode is selected or not. If the free play mode has 
been selected, the processing for the free play mode 
starts in step 8511. The melody selection screen is 
switched to the free play mode screen, and the proce- 
dure then returns to step S6 of the main routine shown 
in FIG. 8. 

[01 11] On the other hand, if it is determined in step 
S510 that the free play mode is not selected, it is deter- 
mined in step S512 whether window closing (or mode 
clear) is selected or not. If window closing is selected 
the window is closed in step S513, and then the normal 
game processing is performed. The procedure then 
returns to step S6 of the main routine shown in FIG. 8. 
[01 1 2] On the other hand, when it is determined in 
step S501 that the melody selection screen is not dis- 
played, it is determined in step S520 of FIG. 12 whether 
the check mode is being executed. If it is determined 
that the check mode is being executed, it is determined 
m step S521 whether auto play is being executed or not 
If it is determined that auto play is not being executed it 
is determined in step S522 whether the controller 40 is 
operated for play or not. that is, whether any push-but- 
ton switch (or joystick) assigned for sound input is 
pressed or not. If the controller 40 is operated for play, 
the sound corresponding to the operated push-button is 
determined in step S523 based on the data inputted by 
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the controller 40. Specifically, the specified sound or 
tone is detected based on the push-button switch and/or 
the data of the tilt amount of the joystick 45 stored in the 
control pad data area 1 56 of the RAM 1 5. 
[0113] In the following S524, in order to display a 
musical note patterned on a fairy as shown in FIG. 18 
on the musical score, a note symbol (object) is regis- 
tered in the display list. Specifically in order to display 
the note object at the position of the tone corresponding 
to the sound detected in step S523 on the score, the 
note object is registered in the display list. For example, 
when the sound input mode is selected by the operation 
of the controller 40, objects for displaying images shown 
in FIG. 15 (for example, a plurality of objects for display- 
ing the score on top of the screen, operation guide on 
the bottom of the screen, the player-object playing the 
ocarina according to sound input operation by the player 
in the middle of the screen) or objects shown in FIG. 16 
(for example, a plurality of objects for displaying images 
indicative of auto play without operation guide, which 
are different from the images in FIG. 15) are registered 
in the display list. At this time, a score such as shown in 
FIG. 1 7 is displayed on a note displaying part located on 
top of the screen. At the position corresponding to the 
note to be inputted, a fairy symbol as shown in FIG. 18 
(a) is displayed for prompting the player to key input, ff 
the predetermined tone is inputted through key input 
operation, the screen indicates that correct key input is 
performed, as shown in FIG. 18(b). When key input is 
not performed within a predetermined time, the screen 
indicates as such shown in FIG. 18 (c). Therefore, 
object data to achieve such display is registered in the 
display list. The drawing processing is performed in step 
S7 based on such registration in the display list when 
the procedure returns to the main routine after the 
processing in step S528, which will be described later. 
Consequently, in step S9, the image shown in the draw- 
ing is displayed on the CRT 30. Further, the data of the 
detected sound is registered in the audio list. 
[0114] Next, in step S525, the tone specified by the 
operation of the controller 40 is compared with a tone of 
an On-th sound in the melody data stored in the melody 
data register R1. The comparison result is stored in the 
sound check register R4. For example, when the speci- 
fied tone coincides with the stored tone, "1 " is registered 
in a bit of the sound check register R4 according to the 
order of sounds. Otherwise, "0" is registered therein. 
The comparison result may be stored as such that "1 " is 
written in the sound check flag F1 when all specified 
tones coincide with the stored tones, while "0" is written 
therein if even a single sound is not correct. 
[0115] Next, in step S526, the storage value of the 
sound number register R2 is incremented by 1 to be 
rewritten as the incremented value. In other words, a 
calculation On = On + 1 is performed, and the latest cal- 
culation result is stored as a new sound number On. In 
the following step S527, it is determined whether the 
storage value On in the sound number register R2 is 



larger than a predetermined number of sounds ("10", for 
example). If it is determined that the storage value is 
larger, the auto play processing is performed in step 
S528. The procedure then returns to step S6 of the 

5 main routine in FIG. 8. 

[0116] On the other hand, if it is determined in the 
above step S522 that the controller 40 is not operated 
for play, it is determined in step S529 whether a prede- 
termined time has elapsed or not. If the predetermined 

io time has elapsed, the procedure advances to step 
S526. Otherwise, the procedure returns to the main rou- 
tine in FIG. 8. The reason for determining whether the 
predetermined time has elapsed or not is for the proce- 
dure to advance to input processing for switches except 

is the sound switches. If the player did not press any push- 
button switch within the predetermined time (five sec- 
onds, for example), it is assumed that sound is not 
inputted. 

[01 17] On the other hand, when it is determined in 
20 above step S521 that auto play is being executed, the 
auto play processing (refer to FIG. 19) is performed in 
step S530 based on the check result. The auto play 
processing is next described in detail with reference to 
FIG. 19. 

25 [0118] It is determined in step S531 whether auto 
play ends or not. If not end, the auto play processing is 
performed in step S532. Specifically, the musical score 
is first cleared. Then, based on the tone data temporar- 
ily stored in the input tone register R4, the note symbols 

30 (objects) are registered in the display list in order to be 
displayed at the positions corresponding to first to last 
inputted sounds. Also, the audio data corresponding to 
these tones is registered in the audio list. 
[01 19] As a result, as shown in FIG. 17, symbols (a 

35 down-pointing triangle, a left-pointing triangle, A, a right- 
pointing triangle, and an up-pointing triangle) indicative 
of the switches (47Cd, 47CI, 47A, 47r, and 47u) corre- 
sponding to the sounds to be inputted are displayed on 
the score. In this state, when a correct switch is oper- 

40 ated, the symbol (A, for example) corresponding to that 
switch is displayed (refer to FIG. 18(b)), and its sound is 
produced. When an incorrect switch is operated, the 
next sound is processed. If no play operation is present 
in the above step S522 and if it is determined in step 

45 S529 that the predetermined time has elapsed, the 
processing in steps S524 and S525 is not performed. 
Therefore, nothing is displayed (refer to FIG. 18(c)) and 
no sound is produced. 

[0120] On the other hand, if it is determined in step 
so S531 that auto play has ended, it is determined in step 
S533 whether the tones inputted by the player are all 
correct or not. This coincidence determination is made 
by comparing the data stored in the input tone register 
R3 with the data stored in the melody data register R1 . 
55 This determination may also be made by determining 
whether every bit of data stored in the sound check reg- 
ister indicates "1 " or not, or the sound check flag F1 indi- 
cates "V* or not. Then, when it is determined that the 
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tones are correct, coincidence processing is performed 
in step S534. As the coincidence processing, predeter- 
mined object data may be registered in the display list 
for displaying that the correct tones have been inputted 
or predetermined audio data may be registered in the s 
audio list for playing music such as a fanfare. In the fol- 
lowing step S535, it is determined whether the coinci- 
dence processing has ended or not. If it is determined 
that the processing has ended, the game processing 
starts in step S536 in response to the input of N-th mel- 10 
ody. For example, the coordinate position of the player- 
object in three-dimensional space is calculated after the 
player-object moves to the place (in the example of FIG. 
13, any one of the first to third places) corresponding to 
the melody selected in the melody selection screen of 1S 
FIG. 14. Accordingly, the place after move is displayed. 
[01 21 ] Therefore, the player-object is warped to the 
place that is different from the place where the player- 
object was before melody input, and displayed in front of 
the background objects, if it is determined in step S535 20 
that the coincidence processing has not ended, the pro- 
cedure returns to step S6 of the main routine in FIG. 8. 
[01 22] On the other hand, in step S533, if it is deter- 
mined in step S533 that the condition "all the tones 
inputted by the player are correct" is not satisfied, the 2s 
procedure advances to step S537. In step S537, "r is 
set in the sound number register R2 (On = 1), prompting 
the player to operate sound input again. Then, the pro- 
cedure returns to step S6 of the main routine. 
[0123] If it is determined in the above step S520 30 
that the check mode is not being executed, it is deter- 
mined in step S540 whether the free play mode is being 
executed or not. If being executed, the free play 
processing is performed. The free play processing is 
shown in FIG. 20 in detail. 35 
[0124] That is, in step S551, based on the data 
stored in the control pad data area 156 of the RAM 15, 
the push-button switch currently being pressed is 
detected. Next, in step S552, the tone corresponding to 
the push-button switch is detected, and the correspond- 40 
ing tone data is generated. Next, it is determined in step 
S553 whether the detected switch is an F button (switch 
47R) or not. If the F button is being pressed, the 
processing for raising the tone in pitch by a semitone is 
performed. Otherwise, the procedure skips step S554 to 45 
advance to step $555. This sharpening processing is 
the processing for changing the tone data so that the 
tone corresponding to the operated switch is raised in 
pitch by a semitone. For example, if the player has 
selected the tone of "la", the tone data for generating the so 
sound having the frequency of 440 Hz is generated. If 
the switch 47R is pressed, the tone data is changed into 
the tone data for generating the sound of 440 x 2 A 
(1/12) Hz, which is a semitone higher than the original 
tone. Note that the symbol represents raising the 55 
value before the symbol to (the following-value inside 
the parentheses) -th power. 

[0125] In the following step S555, it is determined 



whether the joystick 45 is operated forward or backward 
(for example, whether the Y-axis counter 444Y counts 
the tilt of the joystick 45 or not). If it is determined that 
the joystick 45 is operated forward or backward, the 
tone data is changed to change the tone according to 
the tilt angle of the joystick 45. By way of example, when 
the joystick 45 is at a neutral position (the home position 
at the center), the tone is based on the push-button 
switch. When the joystick 45 is tilted forward to maxi- 
mum, the tone is raised in pitch by a whole tone (or one 
tone). When the joystick 45 is tilted backward to maxi- 
mum, the tone is lowered in pitch by a whole tone. When 
the joystick 45 is tilted forward or backward but not to 
maximum, the tone is varied to be raised or lowered 
within a range of one tone according to its tilt angle. 
More specifically, the tone may be raised or lowered by 
a cent (a unit of tone; 2 * (1/200)), which is obtained by 
dividing a whole tone by 200. However, since the Y-axis 
counter 444Y detects the tilt angle of the joystick 45 with 
the count value ranging from 0 to 64, the tone cannot be 
divided by 200. Therefore, when the joystick 45 is tilted 
forward, the frequency of the tone is multiplied by (1 
cent) A (200/64xY) to raise the tone every time the abso- 
lute count value Y varies. On the contrary, when the joy- 
stick 45 is tilted backward, the frequency of the tone is 
divided by (1 cent) * (200/64 xY) every time the abso- 
lute count value X varies. Now, assuming that the tone 
H la" (440 Hz) is selected, the tone data of the tone "la" is 
changed into tone data of 440 x ((2 A (1/200)) A (200/64 

[0126] In other words, by raising or lowering the 
tone specified by the push-button switch within a range 
of a whole tone according to the amount of tilt specified 
by the joystick 45, frequency data of the changed tone is 
generated, and then written and stored in the audio list 
158 (steps S554, S556, S558, and S560) Music data 
inputted by repeating the above steps is read at prede- 
termined cycle in the audio processing of FIG. 25, which 
will be described later, and produced as music. 
[01 27] in stead of changing the tone data within the 
range of a whole tone, the tone data may be raised or 
lowered by a semitone when the joystick 45 is at a posi- 
tion within a predetermined range between the neutral 
position and the maximum forward or backward tilt. Fur- 
ther, the push-button switch may specify two consecu- 
tive tones as a unit. In this case, the joystick 45 can 
specify a tone within the range of two tones (for exam- 
ple, a semitone to a whole tone and a half, within a 
range from a position a little away from the neutral posi- 
tion to the maximum tilt angle). 

[01 28] After step S556 or if it is determined in step 
S555 that the joystick 45 is not operated forward or 
backward, the procedure advances to step S557. It is 
determined in step S557 whether the joystick 45 is oper- 
ated rightward or leftward (that is. whether the X-axis 
counter 444X counts the tilt amount of the joystick 45) 
If it is determined that the joystick 45 is operated right- 
ward or leftward, the processing for changing a depth 
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value o1 vibrato of the tone data according to the tilt 
angle toward right or left of the joystick 45 is performed 
in step S558. For example, when the joystick 45 is at the 
neutral position, the sound is not vibrated. Then the joy- 
stick 45 is tilted rightward or leftward to maximum, the 
sound is vibrated most deeply. When the joystick 45 is 
tilted between the neutral position and the maximum tilt 
position, the depth value is increased or decreased 
according to the tilt angle. In the present embodiment, 
to vibrate in four stages, the count value (X: absolute 
value) ranging 0 to 64 is changed, and the depth value 
is changed accordingly. More specifically, when the joy- 
stick 45 is tilted leftward (or rightward), the depth value 
is set to 1 .001 807 A (X/4). Each numerical value and set 
value is defined through experiments to make comforta- 
ble sound. Now assuming that the user selects the tone 
"la" (440 Hz), the tone data of the tone "la" is changed 
into tone data of the tone subjected to vtorato with its 
frequency being raised or lowered (vibrated) within a 
range between 440 * (depth value =1.001807 A (X/4)) 
and 440/(depth value =1 001807 A (X/4)). 
[0129] After step S558 or rf it is determined in step 
S557 that the joystick 45 is not operated rightward or 
leftward, the procedure advances to step S559. It is 
determined in step S559 whether the push-button 
switch being pressed that was detected in step S551 is 
a G button (switch 47Z) or not. If the push-button switch 
being pressed is the G button, volume data for increas- 
ing the volume by 1 .4 times is generated in step S560 
so that the volume is increased with its tone left 
unchanged. After step S560 or rf it is determined in step 
S559 that the G button is not pressed, the procedure 
returns to step S6 of the main routine in FIG. 8. 
[0130] The tone data and volume data generated 
as described above are registered in the audio list as 
sound data. Such sound data is outputted in the audio 
processing step S8 and the audio output step $10, 
which will be described later. 

[0131 ] On the other hand, if it is determined in step 
S540 that the free play mode is not being executed, it is 
determined in step S570 whether the game mode is 
being executed or not. When the game mode is being 
executed, the game processing is performed in step 
S580. 

[0132] The details in the game processing are illus- 
trated in a subroutine flow chart shown in FIG. 21. That 
is, in step S581, the position of the player-object is 
detected. Next, it is determined in step 582 whether the 
player-object is at a position where the score of a warp 
melody is to be displayed, ff it is determined that the 
player-object is at such position, a notice board object is 
registered in the display list, for example, in step S583, 
in order to display the scores of predetermined melo- 
dies on a notice board: Also the tone data correspond- 
ing to the melodies displayed on the notice board is 
written in the work memory area 157 of the RAM 15. As 
a result, an image as shown in FIG. 22 is displayed. 
Then, the melodies are registered as available melo- 



dies, and displayed as shown in FIG. 14. After step 
S583 or if it is determined in step S582 that the player- 
object is not at the display position, the procedure 
advances to step S584. 

5 [0133] In step S584, it is determined whether the 
player-object is at a predetermined recording place (a 
position where the sound played by the player is to be 
recorded) or not. If it is determined that the player-object 
is at the predetermined recording place, processing of 

io recording the sound played by the player is executed in 
step S585. In the game assumed in the present embod- 
iment, the player-object is instructed to play music when 
the player-object meets a specific person, object, or the 
like, for example. In the recording processing of step 

75 S585, if the player performs operation for free play (refer 
to the description of FIG. 20) according to the instruc- 
tion, the data of the melody to be played is stored in the 
RAM 15. 

[0134] The details of the recording processing are 
20 shown in FIG. 23. That is, in step S586, it is determined 
whether a1/20 second has elapsed since the previous 
recording. If elapsed, data stored in the control pad data 
area 156 (all data or data related to sound) is stored in 
the sound memory area 1 55 as recording data in step 
25 S587. Then, or after it is determined in step S586 that a 
1/20 second has not elapsed, the procedure advances 
to step 588 (refer to FIG. 21). 

[0135] It is determined in step S588 whether the 
player-object is at a predetermined sound check place 

30 or not. When the player-object is at such place, check 
processing of the sound played by the player is exe- 
cuted in step S589. This check processing is similar to 
the above described check processing in steps S520 to 
S530 except that the melody to be checked with the 

35 played melody is "the melody recorded by the player" 
instead of "the melody selected by the player". There- 
fore, description of this check processing is omitted 
herein. After step S589 or if it is determined in step 
S588 that the player-object is not at the sound check 

40 place, the procedure advances to step S590. 

[0136] It is determined in step S590 whether the 
player-object is at a place (or position) where sound is to 
be reproduced. When the player-object is at such place, 
the processing of arranging the sound data based on 

45 the recorded controller data is executed in step S591 . 
This arrangement processing includes processing of 
adding musical characteristics of other musical instru- 
ments except the instrument played by player, or 
processing of changing rhythms according to the mood 

so of the scene. Next, in step S592, sound setting process- 
ing is executed. This processing is to mix and register 
the music data created through the arrange processing 
and other sound data in the audio list. With the sound 
setting processing, the music inputted (composed) by 

55 the player can be generated as BGM during the game 
and also used as a cry of an animal. Next, other game 
processing not performed in the above steps S581 to 
S592 (such as, processing for a fight between the 
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player-object and the enemy and processing for charac- 
ter display) is performed in step S593. 
[0137] Next, the operation of a subroutine of the 
above described drawing processing (step S7) is 
described with reference to FIG. 24. First, in step S70 1 , s 
coordinate transformation processing is performed 
under the control of the RCP 12. In this coordinate 
transformation processing, each coordinate data of a 
plurality of polygons of the moving object such as ene- 
mies, the player and friends and the still object such as 10 
background stored in the image data area 154 of the 
RAM 15 is transformed into coordinates from a camera 
viewpoint. Specifically, in order to obtain an image from 
the viewpoint of the camera, each polygon data con- 
structing a plurality of moving objects and still objects of is Claims 
absolute coordinates is transformed into data of camera 
coordinates. Next, in step S702, drawing processing is 1. 
performed in the frame memory. This processing is per- 
formed by writing color data determined based on the 
texture data onto each of triangular surfaces construct- 
ing each object specified by polygon coordinates, which 
are camera coordinates obtained through the above 
transformation, for each dot of the frame memory area 
152. At this time, in order to display frontward (near) 
objects with priority based on the depth data for each 
polygon, color data of the near objects is written. Also, 
depth data corresponding to the dot in which the color 
data is written is written in the corresponding address of 
the 2 buffer area 153. Then, the procedure returns to 
step S8 of the main routine in FIG. 8. 
[0138] The operation in steps S701 and S702 are 
performed for each frame within a predetermined time 
and for each polygon constructing the plurality of 
objects to be displayed on one screen in sequence. The 
operation is repeated until all objects to be displayed on 
one screen have been processed. 
[0139] Next, the operation of a subroutine of the 
above described audio processing (step S8) is 
described with reference to FIG. 25. First, in step S801 , 
it is determined whether the audio flag is on or not. 
When the audio flag is on, the audio data stored in the 
audio list 158 is read in step S802, and sampled audio 
digital data to be reproduced within one frame (1/60 
second) is outputted to a buffer (not shown). Next, in 
step S803, the audio generator circuit 16 converts the 
digital data stored in the buffer into analog signals, and 
then sequentially outputs these signals to the speaker. 
Then, the procedure returns to step S9 of the main rou- 
tine in FIG. 8, and the processing in steps S9 to S12 is 
performed. 

[0140] Note that a plurality of frequency data corre- 
sponding to music inputted through the operation of the 
controller 40 are registered in the audio list 158 during 
the auto play processing shown in FIG. 19 or the free 
play processing shown in FIG. 20 as described above, ss 
Therefore, such frequency data is sequentially read 
from the audio list 158 in a predetermined cycle (steps 
S801, S802), converted into analog signals during this 



audio processing, and, as a result, produced as music. 

INDUSTRIAL APPLICABILITY 

[0141] As described above, the sound generating 
device according to the present invention is preferably 
applied to electronic equipment such as video game 
devices, personal computers, and electronic musical 
instruments. Especially when used for video game 
devices, the present sound generating device can 
achieve a video game that is rich in variety and much 
fun by using inputted music information with relation to 
the progress of the game. 
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A sound generating device to which sounds of dif- 
ferent tones are inputted and generating the input- 
ted sounds by specifying the tones with a controller 
having a plurality of push-button switches and an 
analog joystick capable of selecting among a plural- 
ity of positions, comprising: 

push-button detection means detecting one of 
said plurality of push-button switches that is 
pressed; 

tone selection means selecting a tone corre- 
sponding to the push-button detected by said 
push-button detection means; 
tilt amount detection means detecting an 
amount of tilt of said analog joystick, 
frequency generation means generating a fre- 
quency corresponding to the tone selected by 
said tone selection means with or without 
change, based on the amount of tilt detected by 
said tilt amount detection means and the push- 
button switch detected by said push-button 
detection means; and 

audio signal generation means generating a 
signal of a sound of the tone corresponding to 
the frequency generated by said frequency 
generation means. 

The sound generating device according to claim 1 
wherein 

when said tilt amount detection means does 
not detect the amount of tilt of said analog joy- 
stick, said frequency generation means gener- 
ates the frequency corresponding to the tone 
selected by said tone selection means without 
change, and 

when said tilt amount detection means detects 
the amount of tilt of said analog joystick, said 
frequency generation means generates the fre- 
quency corresponding to the tone selected by 
said tone selection means with change accord- 
ing to the detected amount of tilt. 
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3. The sound generating device according to claim 1, 
wherein 

said frequency generation means comprises: 

5 

frequency data generation means generat- 
ing frequency data corresponding to the 
push-button switch of the tone selected by 
said tone selection means; 
frequency data storage means temporarily io 
storing a plurality of frequency data; and 
read/write means reading the frequency 
data stored in said frequency data storage 
means or writing the frequency data gen- 
erated by said frequency data generation 75 
means in said frequency data storage 
means, 

when said tilt amount detection means does 
not detect the amount of tilt of said analog joy- 20 
stick, said read/write means writes in said fre- 
quency data storage means a digital value 
equivalent to the frequency corresponding to 
the tone selected by said tone selection 
means, as the frequency data; and 25 
when said tilt amount detection means detects 
the amount of tilt of said analog joystick, said 
read/write means writes in said frequency data 
storage means a digital value equivalent to a 
frequency obtained by changing the frequency 30 
corresponding to the tone selected by said tone 
selection means according to the detected 
amount of tilt, as the frequency data. 

4. The sound generating device according to claim 1 , 35 
wherein 

said frequency generation means 

raises the frequency of the tone within a 40 
predetermined tone range as said analog 
joystick is tilted to one direction; and 
lowers the frequency of the tone within a 
predetermined tone range as said analog 
joystick is tilted to another direction. 45 

5. The sound generating device according to claim 1, 
further comprising: 

vibrato means for changing a depth value of so 
vibrato according to the amount of tilt detected 
by said tilt amount detection means, 
said frequency generation means generates a 
frequency corresponding to the tone selected 
by said tone selection means with vibrato ss 
added thereto based on the depth value from 
said vibrato means. 
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6. A sound generating device to which sounds of dif- 
ferent tones are inputted and generating music 
based on the inputted sounds by specifying the 
tones with a controller having a plurality of push- 
button switches and an analog joystick capable of 
selecting among a plurality of positions, comprising: 

push-button detection means detecting one of 
said plurality of push-button switches that is 
pressed; 

tone selection means selecting a tone corre- 
sponding to the push-button detected by said 
push-button detection means; 
tilt amount detection means detecting an 
amount of tilt of said analog joystick; 
frequency data generation means generating 
frequency data corresponding to the tone 
selected by said tone selection means with or 
without change, based on the amount of tilt 
detected by said tilt amount detection means 
and the pressed push-button switch detected 
by said push-button detection means; 
frequency data storage means temporarily 
storing a plurality of frequency data; 
write means periodically and sequentially writ- 
ing the frequency data generated by said fre- 
quency data generation means in said 
frequency data storage means; 
read means for sequentially reading the fre- 
quency data stored in said frequency data stor- 
age means; and 

audio signal generation means generating an 
audio signal having a frequency corresponding 
to the frequency data read by said read means. 

7. The sound generating device according to claim 6, 
wherein 

said read means repeatedly reads the fre- 
quency data of a predetermined time period 
stored in said frequency data storage means to 
generate music composed by a player as BQM. 

8. A video game device displaying an image on a dis- 
play device and producing sound from a speaker by 
executing a game program, comprising: 

operation means having a plurality of push-but- 
ton switches for instructing motion of a player- 
object on a screen of said display device, and 
an analog joystick capable of selecting among 
a plurality of positions and for instructing a 
moving direction of the player-object through 
operation; 

player-object image data generation means 
generating data for displaying an image of said 
player-object; 
non-player-object image data generation 
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means generating data for display an image of 
an object except said player-object; 
push-button detection means detecting one of 
said plurality of push-button switches that is 
pressed; 5 
tone selection means selecting a tone corre- 
sponding to the push-button detected by said 
push-button detection means; 
tilt amount detection means detecting an 
amount of tilt of said analog joystick; w 
frequency data generation means generating 
frequency data corresponding to the tone 
selected by said tone selection means with or 
without change, based on the amount of tilt 
detected by said tilt amount detection means is 
and the push-button switch detected by said 
push-button detection means; 
frequency data storage means temporarily 
storing a pluralrty of frequency data; 
write means periodically and sequentially writ- so 
ing the frequency data generated by said fre- 
quency data generation means in said 
frequency data storage means; 
read means sequentially reading the frequency 
data stored in said frequency data storage 25 
means; 

audio signal generation means generating an 
audio signal having a frequency corresponding 
to the frequency data read by said read means; 
and 30 
display image changing means changing at 
least one of the image data for the player- 
object generated by said player-object image 
data generation means and the image data for 
the non-player-object generated by said non- 35 
player-object image data generation means 
based on the audio signal generated by said 
audio signal generation means to change at 
least one of display states of the player-object 
and the non-player-object. 40 

9. The video game device according to claim 8, 
wherein 

said display image changing means changes 45 
the display state of said non-player-object. 

10. The video game device according to claim 9, 
wherein 

so 

said display image changing means changes 
the display state of said non-playerobject by 
moving said player-object to a scene which dif- 
fers from a present scene to change a back- 
ground screen of said player-object. ss 

11. The video game device according to claim 8, 
wherein 



said display image changing means changes 
the display state of said player-object. 

1 2. The video game device according to claim 8, further 
comprising: 

predetermined melody determination means 
determining whether a melody based on the 
frequency data sequentially read from said 
read means is a predetermined melody, and 
said display image changing means changes 
at least one of the display states of the player- 
object and the non-player-object in response to 
determination by said predetermined melody 
determination means that the melody is the 
predetermined melody. 

13. The video game device according to claim 12, 
wherein 

said predetermined melody determination 
means temporarily stores melody data inputted 
through operation of said operation means; 
when new melody data is inputted through an 
operation of said operation means a predeter- 
mined time behind, compares the new melody 
data with the melody data previously inputted; 
and when both data has a predetermined rela- 
tion, determines that the melody based on the 
frequency data sequentially read by said read 
means is the predetermined melody. 

14. A video game device displaying an image on a dis- 
play device and producing sound from a speaker by 
executing a game program, comprising: 

operation means operated by a player and hav- 
ing a plurality of push-button switches for 
instructing motion of a player-object on a 
screen of said display device; 
player-object image data generation means 
generating data for displaying an image of said 
player-object; 

non-player-object image data generation 
means generating data for displaying an image 
of an object except said player-object; 
push-button detection means detecting one of 
said plurality of push-button switches that is 
pressed; 

tone selection means selecting a tone corre- 
sponding to the push-button detected by said 
push-button detection means; 
frequency data generation means generating 
frequency data corresponding to the tone 
selected by said tone selection means; 
frequency data storage means temporarily 
storing a plurality of frequency data; 
write means for periodically and sequentially 
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writing the frequency data generated by said 
frequency data generation means in said fre- 
quency data storage means; 
read means for sequentially read the frequency 
data stored in said frequency data storage s 
means; 

audio signal generation means generating an 
audio signal having a frequency corresponding 
to the frequency data read by said read means; 
and 10 
display image changing means, based on the 
audio signal generated by said audio signal 
generation means, changing at least one of the 
display states of the player-object and the non- 
player-object by changing at least one of the is 
image data for the player-object generated by 
said player-object image data generation 
means and the image data for said non-player- 
object generated by said non-player-object 
image data generation means. 20 

15- The video game device according to claim 14, 
wherein 



corresponding to said frequency data; and 
changing at least one of display states of the 
player-objects and the non-player-object by 
changing at least one of the image data for said 
player-object and the image data for said non- 
player-object. 



said display image changing means changes 25 
the display state of said non-player-object by 
moving said player-object to a scene which dif- 
fers from a present scene to change a back- 
ground screen of said player-object. 

30 

16. A recording medium in which a video game pro- 
gram to be executed by an information processing 
device for displaying an image for a game on a dis- 
play device and producing sound for the game from 
a speaker is stored, 35 

said information processing device comprising 
operation means operated by a player and hav- 
ing a plurality of push-button switches for 
instructing motion of a player-object on a 40 
screen of said display device, 
said video game program being for realizing an 
operational environment on said information 
processing device, said program comprising 
the steps of: 45 
generating data for displaying an image of the 
player-object in response to an operation of 
said operation means; 

generating data for displaying an image of an 
object except the player-object (non-player- so 
object) in response to an operation of said 
operation means; 

detecting one of said plurality of push-button 
switches that is pressed and selecting a tone 
corresponding to the pressed push-button: 55 
generating frequency data corresponding to 
the selected tone; 

generating an audio signal having a frequency 
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